I think your methodology is faulty because it's not possible ibex finished in under 1 second. Did you make clean between your measurements?

@rovinski To clarify, I'm not running the full Ibex flow, I've previously finished the flow and running make again just prints out resource usages of each stage. I guess a more clear example would be make DESIGN_CONFIG=... elapsed? This PR just removes some unnecessary overhead from running the Makefile.

There is something pathologically bad about NIX if doing essentially no work takes 50s. @hzeller have observed this?

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 21 days if no further activity occurs. Remove the Stale label or comment to keep it open.

Ping to keep issue active. I haven't looked into the cause for the repeated expansions yet, but I can have a go at it.

I think I have an explanation. Variables marked export are always expanded when any shell is started, so something like

export TEST = $(shell echo "expanded" >> /tmp/expand.txt)
UNRELATED := $(shell echo "unrelated")

writes out

expanded
expanded

to /tmp/expand.txt. Since UNRELATED tries to launch a shell, TEST is expanded, and since it tries to launch a shell, TEST is expanded again. At this point, GNU Make detects the recursion and expands TEST as an empty string, see https://www.gnu.org/software/make/manual/html_node/Shell-Function.html

In the case of OpenROAD, all of OPENROAD_EXE, OPENROAD_CMD, OPENROAD_NO_EXIT_CMD, OPENROAD_GUI_CMD, PYTHON_EXE and YOSYS_EXE are marked export, so any time any 'legitimate' shell is started, at least 48 additional 'accidental' shells are started. I'm not entirely sure why 48 precisely, but that's what I get based on the following simplified situation:

$(shell echo -n > /tmp/expand.txt)

export OPENROAD_EXE = $(shell echo expanded >> /tmp/expand.txt)
export YOSYS_EXE = $(shell echo expanded >> /tmp/expand.txt)
export PYTHON_EXE = $(shell echo expanded >> /tmp/expand.txt)
export OPENROAD_GUI_CMD = $(OPENROAD_EXE)
export OPENROAD_CMD = $(OPENROAD_EXE)
export OPENROAD_NO_EXIT_CMD = $(OPENROAD_EXE)

all:
    wc -l /tmp/expanded.txt

Expanding an envar should be a millisecond operation unless you are executing some command from within the envar.

Running IN_NIX_SHELL=1 make DESIGN_CONFIG=designs/sky130hd/ibex/config.mk nada, so not even matching any rules, starts 6519 shells in total. Just starting and exiting that many shells takes around 6 seconds on my machine, and a lot of the executed commands seem to start subshells of their own. I'm fairly certain this is the crux of the issue.

Ping again, resolved merge conflict that appeared.

Is there something I can do to help get this merged? It's a very minor issue in the grand scheme of things, but still annoying. I believe I've shown what the root cause is (exported deferred variables containing $(shell) invocations causing a compounding number of shells to be executed) and provided a simple fix (re-define the variables as immediately expanded). Is there anything else that would need attention?

/gemini review